Papers with generative language model
GraphNarrator: Generating Textual Explanations for Graph Neural Networks (2025.acl-long)
Copied to clipboard
| Challenge: | Graph representation learning has garnered significant attention due to its broad applications in various domains, such as recommendation systems and social network analysis. |
| Approach: | They propose to use a generative language model to map input-output pairs to explanations reflecting the model’s decision-making process to generate a model that generates pseudo-labels that capture the model's decisions from saliency-based explanations. |
| Outcome: | Extensive experiments show that GraphNarrator produces human-preferred explanations that are faithful, concise, and human-like. |
Addressing Domain Changes in Task-oriented Conversational Agents through Dialogue Adaptation (2023.eacl-srw)
Copied to clipboard
| Challenge: | Recent task-oriented dialogue systems are trained on annotated dialogues, but when domain knowledge changes, the initial model may become obsolete. |
| Approach: | They propose to use an annotated dialogue dataset to train a dialogue model for domain changes . they propose to fine-tune a generative language model on domain changes to reduce performance . |
| Outcome: | The proposed approach reduces performance by 55% by fine-tuning a generative language model on domain changes. |
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to language-based environment manipulation are difficult to generalize across environments. |
| Approach: | They propose a general framework for language-based environment manipulation tasks that can deal with various environments using the same generative language model. |
| Outcome: | The proposed framework achieves new state-of-the-art results on four of the tasks and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks. |
A Generative Language Model for Few-shot Aspect-Based Sentiment Analysis (2022.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models are often used to achieve state-of-the-art results . eval paper shows that generative language model can handle joint and multi-task settings . |
| Approach: | They propose to reformulate extraction and prediction tasks into a sequence generation task . they propose a generative language model with unidirectional attention that learns to accomplish the tasks via language generation . |
| Outcome: | The proposed model outperforms the state-of-the-art in few-shot and full-shot settings. |
On Generative Spoken Language Modeling from Raw Audio (2021.tacl-1)
Copied to clipboard
Kushal Lakhotia, Eugene Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Abdelrahman Mohamed, Emmanuel Dupoux
| Challenge: | Using a set of metrics to evaluate the learned representations, we aim to create a system that learns from natural interactions as infants learn their first language. |
| Approach: | They propose a task of learning acoustic and linguistic characteristics from raw audio and a set of metrics to evaluate the learned representations at acustic, linguistic and encoding levels. |
| Outcome: | The proposed models evaluate the learned representations at acoustic and linguistic levels for both encoding and generation. |
Generative Spoken Language Model based on continuous word-sized audio tokens (2023.emnlp-main)
Copied to clipboard
| Challenge: | Text-based language models outperform character-based models, but speech inputs are 20ms or 40ms-long discrete units. |
| Approach: | They propose a generative language model based on word-size continuous audio tokens . they replace lookup table for lexical types with a Lexical Embedding function . |
| Outcome: | The proposed model is five times more memory efficient than discrete unit GSLMs and is phonetically and semantically interpretable. |
Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speech (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies on generating hate speech/counter narratives have failed to reach high-quality datasets. |
| Approach: | They propose a human-in-the-loop data collection methodology that refines a generative language model iteratively by using its own data from previous loops to generate new training samples. |
| Outcome: | The proposed method is the only expert-based multi-target HS/CN dataset available to the community. |
ODIST: Open World Classification via Distributionally Shifted Instances (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work to achieve open-world classification capability in natural language processing and computer vision focuses on decision boundary finding. |
| Approach: | They propose a method that can create out-of-domain instances from in-domain training instances with the help of a pre-trained generative language model. |
| Outcome: | The proposed method can create out-of-domain instances from the in-domain training instances with the help of a pre-trained generative language model. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Towards Imperceptible Document Manipulations against Neural Ranking Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to detect vulnerabilities in neural ranking models often introduce noticeable errors and require a well-imitated surrogate NRM to guarantee the attack effect. |
| Approach: | They propose a framework called Imperceptible DocumEnt Manipulation to produce adversarial documents that are less noticeable to both algorithms and humans. |
| Outcome: | The proposed framework outperforms strong baselines while maintaining fluency and correctness of the target documents. |
Evaluating the Potential of Language-family-specific Generative Models for Low-resource Data Augmentation: A Faroese Case Study (2024.lrec-main)
Copied to clipboard
| Challenge: | generative language models have shown promising results for translation in zero, one, and fewshot learning settings, among other types of tasks. |
| Approach: | They propose to prompt a generative language model for the Nordic languages for Faroese to English translation in a zero, one, and few-shot setting and challenge its Farose language understanding capabilities on a small dataset. |
| Outcome: | The proposed model can translate Faroese to English in a zero, one, and few-shot setting and then use it to create an annotated Farose semantic textual similarity (STS) dataset. |
IDEM: The IDioms with EMotions Dataset for Emotion Recognition (2024.lrec-main)
Copied to clipboard
Alexander Prochnow, Johannes E. Bendler, Caroline Lange, Foivos Ioannis Tzavellos, Bas Marco Göritzer, Marijn ten Thij, Riza Batista-Navarro
| Challenge: | idiomatic expressions are used in everyday language and typically convey affect, i.e., emotion. |
| Approach: | They present a dataset of idiom-containing sentences that were generated and labelled with any one of 36 emotion types using a generative language model. |
| Outcome: | The proposed method achieves an agreement rate of 62% on the IDioms with EMotions dataset, with human validation by two independent annotators. |